Seasoned analytics practitioners invariably have many “you-live-and-learn” experiences. They commonly include cases in which the problem presented to them is not the real one to be solved.
Unfortunately, in practice, the problem presented is often blindly accepted as the one to be solved. This leads to solving the wrong problem. Disconnects ensue. And the consequences may not be trivial.
A key precondition to addressing the needs is correctly understanding what the true problem is. However, everyone is so focused on the idea of solving. Without giving it much thought, everyone happily accepts solving a problem rather than solving the problem.
Wrong problem: a real (obvious) case
A collections manager at a bank engaged an analytical services provider. His need was to collect smarter before the accounts reached 30 days past due. He thought he could collect “more.” This was a classic case of wanting to incorporate analytics to do better. It is what every analytics practitioner wants everyone else to do!
Prioritization is a common need in collections. At that time, his team was calling past-due customers in no specific order.
The provider proposed developing a custom scoring model to order past-due customers. The idea was to prioritize the calling activities by the likelihood of payment. Makes sense.
(We could debate whether this represents best-practice actions in collections, but let’s table that discussion here. Also, while the amount owed is often important in collections, the amounts in this case were relatively uniform).
Both sides are happy; the contract is signed. The initial exploratory data analysis begins. The provider finds out the bank collects nearly 100% of its past-due accounts within the first 30 days.
The last time I checked, it was still impossible to collect more than 100% of what is owed. Collecting more could not possibly have been the real problem. But no one thought to question.
The client stated the problem as he perceived it. It is often up to analytics practitioners to identify and solve the true problem rather than accepting the one presented.
I still get a chuckle from time to time.
Understanding the problem vs. solving the problem
In school, we rarely learn much outside of the analytical techniques used. We do not explicitly learn how to solve problems. Instead, we tend to be natural problem solvers and simply figure out leverage the technical stuff we learn in school.
(It is important to recognize this is not a given in all analytics practitioners. Not everyone is a natural problem solver. However, it does not mean these analytics practitioners are better or worse. Rather, it is that they are of a different type. There are analytics roles for them, just different from what many people think they need.)
Analytics practitioners are generally very good at solving a well-defined problem. However, few are very good at getting to the right problem from the one presented and making it well-defined.
Competencies in solving problems do not equate to competencies in defining and analyzing the problems themselves. The problem presented almost always needs to be refined to the right one based on the situation and the context. This is a very different activity from analyzing data for solving a well-defined problem. It is also a critical need that continues to be underappreciated.
Rarely do I come across analytics practitioners who do this well. Often, it is not even part of what they want to do. However, it does not mean they are any better or worse at “analytics”—just different.
Who identifies the real problem?
I am not advocating that all analytics practitioners be experts at diagnosing problems. I am, however, advocating we all develop a greater appreciation for the process of deriving well-defined true problems.
All analytics practitioners should pick up basic problem diagnosis techniques. Curiously, coursework in analytics generally does not address how to analyze the problems themselves. You do not find things like root cause analysis and how that applies in the context of analytics.
Those who are very good at understanding and defining the true problems are very adept at asking the “why” questions. From my experience, they possess not just the ability but also the enthusiasm for understanding that there is more to it than meets the eye. They are comfortable foregoing all assumptions to get to the heart of the matter. They thrive in highly ambiguous situations. They love the challenge of connecting the dots to formulate the right questions. A big part of this is not just the level but also the type of curiosity. All without developing trust issues!
The reality is that they are hard to find. They are also hard to evaluate. It escapes even the most technically accomplished analytics experts. I have seen so many organizations make less-than-perfect decisions here.
Again, different—not better or worse.
Making sure the problem is the right one
No cutting-edge analytics can overcome a poorly defined problem. At best, it is the proverbial “spray and pray.” Making sure the problem is the right one is the critical first step toward enjoying the fruits of the wizardry that is analytics.
Again, not all analytics practitioners are great at this, nor does it usually make sense to try to make them great at it. Instead, enlist the right resources to help you get to the true problem and make it well-defined.
They are not just thinkers; they do analytics. But they are really good at the kind of analytics that involves very different thinking from that used in analytics development.
Without them, you solve the wrong problem every time. The resulting solution is suboptimal at best. It may be a waste of time and money, a net negative, or even a source of new risks in some cases.
Experiencing frustrations with analytics? Among others, a good place to start is looking at how the problems are being defined. Get this nailed down, and things naturally start to go in the right direction.
